Ontological and Non-Ontological Resources for Associating Medical Dictionary for Regulatory Activities Terms to SNOMED Clinical Terms With Semantic Properties.

Background: Formal definitions allow selecting terms (e.g., identifying all terms related to "Infectious disease" using the query "has causative agent organism") and terminological reasoning (e.g., "hepatitis B" is a "hepatitis" and is an "infectious disease"). However, the standard international terminology Medical Dictionary for Regulatory Activities (MedDRA) used for coding adverse drug reactions in pharmacovigilance databases does not beneficiate from such formal definitions. Our objective was to evaluate the potential of reuse of ontological and non-ontological resources for generating such definitions for MedDRA. Methods: We developed several methods that collectively allow a semiautomatic semantic enrichment of MedDRA: 1) using MedDRA-to-SNOMED Clinical Terms (SNOMED CT) mappings (available in the Unified Medical Language System metathesaurus or other mapping resources, e.g., the MedDRA preferred term "hepatitis B" is associated to the SNOMED CT concept "type B viral hepatitis") to extract term definitions (e.g., "hepatitis B" is associated with the following properties: has finding site liver structure, has associated morphology inflammation morphology, and has causative agent hepatitis B virus); 2) using MedDRA labels and lexical/syntactic methods for automatic decomposition of complex MedDRA terms (e.g., the MedDRA systems organ class "blood and lymphatic system disorders" is decomposed in blood system disorders and lymphatic system disorders) or automatic suggestions of properties (e.g., the string "cyclic" in preferred term "cyclic neutropenia" leads to the property has clinical course cyclic). Results: The Unified Medical Language System metathesaurus was the main ontological resource reusable for generating formal definitions for MedDRA terms. The non-ontological resources (another mapping resource provided by Nadkarni and Darer in 2010 and MedDRA labels) allowed defining few additional preferred terms. While the Ci4SeR tool helped the curator to define 1,935 terms by suggesting potential supplemental relations based on the parents' and siblings' semantic definition, defining manually all MedDRA terms remains expensive in time. Discussion: Several ontological and non-ontological resources are available for associating MedDRA terms to SNOMED CT concepts with semantic properties, but providing manual definitions is still necessary. The ontology of adverse events is a possible alternative but does not cover all MedDRA terms either. Perspectives are to implement more efficient techniques to find more logical relations between SNOMED CT and MedDRA in an automated way.

Semantic Queries Expedite MedDRA Terms Selection Thanks to a Dedicated User Interface: A Pilot Study on Five Medical Conditions.

Background: Searching into the MedDRA terminology is usually limited to a hierarchical search, and/or a string search. Our objective was to compare user performances when using a new kind of user interface enabling semantic queries versus classical methods, and evaluating term selection improvement in MedDRA. Methods: We implemented a forms-based web interface: OntoADR Query Tools (OQT). It relies on OntoADR, a formal resource describing MedDRA terms using SNOMED CT concepts and corresponding semantic relations, enabling terminological reasoning. We then compared time spent on five examples of medical conditions using OQT or the MedDRA web-based browser (MWB), and precision and recall of the term selection. Results: OntoADR Query Tools allows the user to search in MedDRA: One may enter search criteria by selecting one semantic property from a dropdown list and one or more SNOMED CT concepts related to the range of the chosen property. The user is assisted in building his query: he can add criteria and combine them. Then, the interface displays the set of MedDRA terms matching the query. Meanwhile, on average, the time spent on OQT (about 4 min 30 s) is significantly lower (-35%; p < 0.001) than time spent on MWB (about 7 min). The results of the System Usability Scale (SUS) gave a score of 62.19 for OQT (rated as good). We also demonstrated increased precision (+27%; p = 0.01) and recall (+34%; p = 0.02). Computed "performance" (correct terms found per minute) is more than three times better with OQT than with MWB. Discussion: This pilot study establishes the feasibility of our approach based on our initial assumption: performing MedDRA queries on the five selected medical conditions, using terminological reasoning, expedites term selection, and improves search capabilities for pharmacovigilance end users. Evaluation with a larger number of users and medical conditions are required in order to establish if OQT is appropriate for the needs of different user profiles, and to check if conclusions can be extended to other kinds of medical conditions. The application is currently limited by the non-exhaustive coverage of MedDRA by OntoADR, but nevertheless shows good performance which encourages continuing in the same direction.

Evaluating Twitter as a complementary data source for pharmacovigilance.

BACKGROUND: Social media are currently considered as a potential complementary source of knowledge for drug safety surveillance. Our primary objective was to estimate the frequency of adverse drug reactions (ADRs) experienced by Twitter users. Our secondary objective was to determine whether tweets constitute a valuable and informative source of data for pharmacovigilance purposes, despite limitations on character number per tweet. RESEARCH DESIGN AND METHODS: We selected a list of 33 drugs subject to careful monitoring due to safety concern in France and Europe, and extracted tweets using the streaming API from 30 September 2014 to 5 April 2015. Two pharmacovigilance centers classified these tweets manually as potential ADR case reports. RESULTS: Among 10,534 tweets, 848 (8.05%) implied or mentioned an ADR without meeting the four FDA criteria required for reporting an ADR, and 289 (2.74%) tweets were classified as 'case reports.' Among them 20 (7.27%) tweets mentioned an unexpected ADR and 33 (11.42%) tweets mentioned a serious ADR. CONCLUSIONS: With the use of dedicated tools, Twitter could become a complementary source of information for pharmacovigilance, despite a major limitation regarding causality assessment of ADRs in individual tweets, which may improve with the new limitation to 280 characters per tweet.

The Adverse Drug Reactions from Patient Reports in Social Media Project: Five Major Challenges to Overcome to Operationalize Analysis and Efficiently Support Pharmacovigilance Process.

BACKGROUND: Adverse drug reactions (ADRs) are an important cause of morbidity and mortality. Classical Pharmacovigilance process is limited by underreporting which justifies the current interest in new knowledge sources such as social media. The Adverse Drug Reactions from Patient Reports in Social Media (ADR-PRISM) project aims to extract ADRs reported by patients in these media. We identified 5 major challenges to overcome to operationalize the analysis of patient posts: (1) variable quality of information on social media, (2) guarantee of data privacy, (3) response to pharmacovigilance expert expectations, (4) identification of relevant information within Web pages, and (5) robust and evolutive architecture. OBJECTIVE: This article aims to describe the current state of advancement of the ADR-PRISM project by focusing on the solutions we have chosen to address these 5 major challenges. METHODS: In this article, we propose methods and describe the advancement of this project on several aspects: (1) a quality driven approach for selecting relevant social media for the extraction of knowledge on potential ADRs, (2) an assessment of ethical issues and French regulation for the analysis of data on social media, (3) an analysis of pharmacovigilance expert requirements when reviewing patient posts on the Internet, (4) an extraction method based on natural language processing, pattern based matching, and selection of relevant medical concepts in reference terminologies, and (5) specifications of a component-based architecture for the monitoring system. RESULTS: Considering the 5 major challenges, we (1) selected a set of 21 validated criteria for selecting social media to support the extraction of potential ADRs, (2) proposed solutions to guarantee data privacy of patients posting on Internet, (3) took into account pharmacovigilance expert requirements with use case diagrams and scenarios, (4) built domain-specific knowledge resources embeding a lexicon, morphological rules, context rules, semantic rules, syntactic rules, and post-analysis processing, and (5) proposed a component-based architecture that allows storage of big data and accessibility to third-party applications through Web services. CONCLUSIONS: We demonstrated the feasibility of implementing a component-based architecture that allows collection of patient posts on the Internet, near real-time processing of those posts including annotation, and storage in big data structures. In the next steps, we will evaluate the posts identified by the system in social media to clarify the interest and relevance of such approach to improve conventional pharmacovigilance processes based on spontaneous reporting.

[Automated grouping of terms associated to cardiac valve fibrosis in MedDRA]. / Groupement automatisé de termes liés aux valvulopathies médicamenteuses dans MedDRA.

AIM: To propose an alternative approach for building custom groupings of terms that complements the usual approach based on both hierarchical method (selection of reference groupings in medical dictionary for regulatory activities [MedDRA]) and/or textual method (string search), for case reports extraction from a pharmacovigilance database in response to a safety problem. Here we take cardiac valve fibrosis as an example. METHODS: The list of terms obtained by an automated approach, based on querying ontology of adverse drug reactions (OntoADR), a knowledge base defining MedDRA terms through relationships with systematized nomenclature of medicine-clinical terms (SNOMED CT) concepts, was compared with the reference list consisting of 53 preferred terms obtained by hierarchical and textual method. Two queries were performed on OntoADR by using a dedicated software: OntoADR query tools. Both queries excluded congenital diseases, and included a procedure or an auscultation method performed on cardiac valve structures. Query 1 also considered MedDRA terms related to fibrosis, narrowing or calcification of heart valves, and query 2 MedDRA terms described according to one of these four SNOMED CT terms: "Insufficiency", "Valvular sclerosis", "Heart valve calcification" or "Heart valve stenosis". RESULTS: The reference grouping consisted of 53 MedDRA preferred terms. Our automated method achieved recall of 79% and precision of 100% for query 1 privileging morphological abnormalities, and recall of 100% and precision of 96% for query 2 privileging functional abnormalities. CONCLUSION: An alternative approach to MedDRA reference groupings for building custom groupings is feasible for cardiac valve fibrosis. OntoADR is still in development. Its application to other adverse reactions would require significant work for a knowledge engineer to define every MedDRA term, but such definitions could then be queried as many times as necessary by pharmacovigilance professionals.

OntoADR a semantic resource describing adverse drug reactions to support searching, coding, and information retrieval.

INTRODUCTION: Efficient searching and coding in databases that use terminological resources requires that they support efficient data retrieval. The Medical Dictionary for Regulatory Activities (MedDRA) is a reference terminology for several countries and organizations to code adverse drug reactions (ADRs) for pharmacovigilance. Ontologies that are available in the medical domain provide several advantages such as reasoning to improve data retrieval. The field of pharmacovigilance does not yet benefit from a fully operational ontology to formally represent the MedDRA terms. Our objective was to build a semantic resource based on formal description logic to improve MedDRA term retrieval and aid the generation of on-demand custom groupings by appropriately and efficiently selecting terms: OntoADR. METHODS: The method consists of the following steps: (1) mapping between MedDRA terms and SNOMED-CT, (2) generation of semantic definitions using semi-automatic methods, (3) storage of the resource and (4) manual curation by pharmacovigilance experts. RESULTS: We built a semantic resource for ADRs enabling a new type of semantics-based term search. OntoADR adds new search capabilities relative to previous approaches, overcoming the usual limitations of computation using lightweight description logic, such as the intractability of unions or negation queries, bringing it closer to user needs. Our automated approach for defining MedDRA terms enabled the association of at least one defining relationship with 67% of preferred terms. The curation work performed on our sample showed an error level of 14% for this automated approach. We tested OntoADR in practice, which allowed us to build custom groupings for several medical topics of interest. DISCUSSION: The methods we describe in this article could be adapted and extended to other terminologies which do not benefit from a formal semantic representation, thus enabling better data retrieval performance. Our custom groupings of MedDRA terms were used while performing signal detection, which suggests that the graphical user interface we are currently implementing to process OntoADR could be usefully integrated into specialized pharmacovigilance software that rely on MedDRA.

MedDRA® automated term groupings using OntoADR: evaluation with upper gastrointestinal bleedings.

OBJECTIVE: To propose a method to build customized sets of MedDRA terms for the description of a medical condition. We illustrate this method with upper gastrointestinal bleedings (UGIB). RESEARCH DESIGN AND METHODS: We created a broad list of MedDRA terms related to UGIB and defined a gold standard with the help of experts. MedDRA terms were formally described in a semantic resource named OntoADR. We report the use of two semantic queries that automatically select candidate terms for UGIB. Query 1 is a combination of two SNOMED CT concepts describing both morphology 'Hemorrhage' and finding site 'Upper digestive tract structure'. Query 2 complements Query 1 by taking into account MedDRA terms associated to SNOMED CT concepts describing clinical manifestations 'Melena' or 'Hematemesis'. RESULTS: We compared terms in queries and our gold standard achieving a recall of 71.0% and a precision of 81.4% for query 1 (F1 score 0.76); and a recall of 96.7% and a precision of 77.0% for query 2 (F1 score 0.86). CONCLUSIONS: Our results demonstrate the feasibility of applying knowledge engineering techniques for building customized sets of MedDRA terms. Additional work is necessary to improve precision and recall, and confirm the interest of the proposed strategy.

A Patient Safety Information Model for Interoperability.

Current systems that target Patient Safety (PS) like mandatory reporting systems and specific vigilance reporting systems share the same information types but are not interoperable. Ten years ago, WHO embarked on an international project to standardize quality management information systems for PS. The goal is to support interoperability between different systems in a country and to expand international sharing of data on quality and safety management particularly for less developed countries. Two approaches have been used: (i) a bottom-up one starting with existing national PS reporting and international or national vigilance systems, and (ii) a top-down approach that uses the Patient Safety Categorial Structure (PS-CAST) and the Basic Formal Ontology (BFO) upper level ontology versions 1 and 2. The output is currently tested as an integrated information system for quality and PS management in four WHO member states.

ICHI Categorial Structure: a WHO-FIC Tool for Semantic Interoperability of Procedures Classifications.

Casemix grouping using procedures classifications has become an important use case for health care terminologies. There are so many different national procedures classifications used for Casemix grouping that it is not possible to agree on a worldwide standard. ICHI (International Classification of Health Interventions) is proposing an approach that standardises only the terminologies' model structure. The poster shows the use of the ICHI alpha to replace ICD9 CM Volume 3 in the UNU-CBG International Casemix grouper.

Adverse Drug Reaction Identification and Extraction in Social Media: A Scoping Review.

BACKGROUND: The underreporting of adverse drug reactions (ADRs) through traditional reporting channels is a limitation in the efficiency of the current pharmacovigilance system. Patients' experiences with drugs that they report on social media represent a new source of data that may have some value in postmarketing safety surveillance. OBJECTIVE: A scoping review was undertaken to explore the breadth of evidence about the use of social media as a new source of knowledge for pharmacovigilance. METHODS: Daubt et al's recommendations for scoping reviews were followed. The research questions were as follows: How can social media be used as a data source for postmarketing drug surveillance? What are the available methods for extracting data? What are the different ways to use these data? We queried PubMed, Embase, and Google Scholar to extract relevant articles that were published before June 2014 and with no lower date limit. Two pairs of reviewers independently screened the selected studies and proposed two themes of review: manual ADR identification (theme 1) and automated ADR extraction from social media (theme 2). Descriptive characteristics were collected from the publications to create a database for themes 1 and 2. RESULTS: Of the 1032 citations from PubMed and Embase, 11 were relevant to the research question. An additional 13 citations were added after further research on the Internet and in reference lists. Themes 1 and 2 explored 11 and 13 articles, respectively. Ways of approaching the use of social media as a pharmacovigilance data source were identified. CONCLUSIONS: This scoping review noted multiple methods for identifying target data, extracting them, and evaluating the quality of medical information from social media. It also showed some remaining gaps in the field. Studies related to the identification theme usually failed to accurately assess the completeness, quality, and reliability of the data that were analyzed from social media. Regarding extraction, no study proposed a generic approach to easily adding a new site or data source. Additional studies are required to precisely determine the role of social media in the pharmacovigilance system.

Improvement of Diagnosis Coding by Analysing EHR and Using Rule Engine: Application to the Chronic Kidney Disease.

Coding medical diagnosis in case mix databases is a time-consuming task as every information available in patient records has to be taken into account. We developed rules based on EHR data with the Drools rules engine in order to support diagnosis coding of chronic kidney disease (CKD) in our hospital. 520 patients had a GFR < 60 ml/min as estimated by the Cockroft-Gault formula and corresponded to 429 case mix database entries. We compared stays in which the patient was older than 12 and younger than 65 or 80 at the time of the stay. We concluded that our rules engine implementation may improve coding of CKD for 45.6% of patients with a GFR < 60 ml/min and younger than 65. When patients are older than 65 our rule engine may be less useful for suggesting missing codes of CKD because the estimation of GFR by the Cockroft-Gault formula becomes less reliable as patients get older.

Toward a patient safety upper level ontology.

Patient Safety (PS) standardization is the key to improve interoperability and expand international share of incident reporting system knowledge. By aligning the Patient Safety Categorial Structure (PS-CAST) to the Basic Formal Ontology version 2 (BFO2) upper level ontology, we aim to provide more rigor on the underlying organization on the one hand, and to share instances of concepts of categorial structure on the other hand. This alignment is a big step in the top-down approach, to build a complete and standardized domain ontology in order to facilitate the basis to a WHO accepted new information model for Patient Safety.

Integrated care: an Information Model for Patient Safety and Vigilance Reporting Systems.

Quality management information systems for safety as a whole or for specific vigilances share the same information types but are not interoperable. An international initiative tries to develop an integrated information model for patient safety and vigilance reporting to support a global approach of heath care quality.

Ci4SeR--curation interface for semantic resources--evaluation with adverse drug reactions.

Evaluation and validation have become a crucial problem for the development of semantic resources. We developed Ci4SeR, a Graphical User Interface to optimize the curation work (not taking into account structural aspects), suitable for any type of resource with lightweight description logic. We tested it on OntoADR, an ontology of adverse drug reactions. A single curator has reviewed 326 terms (1020 axioms) in an estimated time of 120 hours (2.71 concepts and 8.5 axioms reviewed per hour) and added 1874 new axioms (15.6 axioms per hour). Compared with previous manual endeavours, the interface allows increasing the speed-rate of reviewed concepts by 68% and axiom addition by 486%. A wider use of Ci4SeR would help semantic resources curation and improve completeness of knowledge modelling.

Automatic annotation of ICD-to-MedDRA mappings with SKOS predicates.

Robust alignments between ICD and MedDRA are essential to enable the secondary use of clinical data for pharmacovigilance research. UMLS makes available ICD-to-MedDRA mappings, but they are only poorly specified, which introduces difficulties when exploited in an automatic way. SKOS vocabulary can help achieve quality and machine-processable mappings. We have developed an algorithm based on several simple rules which annotates automatically ICD-to-MedDRA mappings with SKOS predicates. The method was tested and evaluated on a sample of ICD-10-to MedDRA mappings extracted from UMLS. The algorithm demonstrated satisfying performances, especially for skos:exactMatch properties, which suggests that automatic methods can be used to improve the quality of terminology mappings.

Formalizing MedDRA to support semantic reasoning on adverse drug reaction terms.

Although MedDRA has obvious advantages over previous terminologies for coding adverse drug reactions and discovering potential signals using data mining techniques, its terminological organization constrains users to search terms according to predefined categories. Adding formal definitions to MedDRA would allow retrieval of terms according to a case definition that may correspond to novel categories that are not currently available in the terminology. To achieve semantic reasoning with MedDRA, we have associated formal definitions to MedDRA terms in an OWL file named OntoADR that is the result of our first step for providing an "ontologized" version of MedDRA. MedDRA five-levels original hierarchy was converted into a subsumption tree and formal definitions of MedDRA terms were designed using several methods: mappings to SNOMED-CT, semi-automatic definition algorithms or a fully manual way. This article presents the main steps of OntoADR conception process, its structure and content, and discusses problems and limits raised by this attempt to "ontologize" MedDRA.

Method for mapping the French CCAM terminology to the UMLS metathesaurus.

The French coding system of surgical procedures, the Classification Commune des Actes Médicaux (CCAM), is used in France for DRG databases and fee for services payment. Mapping between CCAM and other clinical procedures terminologies by the means of UMLS metathesaurus is essential in order to increase semantic interoperability between different healthcare terminologies and between different case mix systems. In a previous work the CISMeF team used an automatic approach to map CCAM descriptors to the French part of the UMLS metathesaurus. In another way for the French funded research project InterSTIS, we performed a mapping using MetaMap based on the top level semantic structure descriptors of anatomy and action of CCAM translated from French to English. This paper aims to present this new approach and to compare the results with the previous one. The combination of both approaches significantly improved the coverage of the mapping to 68 % for both descriptors and 95 % for at least one descriptor.

Evaluation of automated term groupings for detecting anaphylactic shock signals for drugs.

Signal detection in pharmacovigilance should take into account all terms related to a medical concept rather than a single term. We built an OWL-DL file with formal definitions of MedDRA and SNOMED-CT concepts and performed two queries, Query 1 and 2, to retrieve narrow and broad terms within the Standard MedDRA Query (SMQ) related to 'anaphylactic shock' and the terms from the High Level Term (HLT) grouping related to 'anaphylaxis'. We compared values of the EB05 (EBGM) statistical test for disproportionality with 50 active ingredients randomly selected in the public version of the FDA pharmacovigilance database. Coefficient of correlation was R(2) = 1.00 between Query 1 and HLT; R(2) = 0.98 between Query 1 and SMQ narrow; R(2) = 0.89 between Query 2 and SMQ Narrow+Broad. Generating automated groupings of terms for signal detection is feasible but requires additional efforts in modeling MedDRA terms in order to improve precision and recall of these groupings.

Modeling patient safety incidents knowledge with the Categorial Structure method.

Following the WHO initiative named World Alliance for Patient Safety (PS) launched in 2004 a conceptual framework developed by PS national reporting experts has summarized the knowledge available. As a second step, the Department of Public Health of the University of Saint Etienne team elaborated a Categorial Structure (a semi formal structure not related to an upper level ontology) identifying the elements of the semantic structure underpinning the broad concepts contained in the framework for patient safety. This knowledge engineering method has been developed to enable modeling patient safety information as a prerequisite for subsequent full ontology development. The present article describes the semantic dissection of the concepts, the elicitation of the ontology requirements and the domain constraints of the conceptual framework. This ontology includes 134 concepts and 25 distinct relations and will serve as basis for an Information Model for Patient Safety.